This disclosure is based upon Japanese Application Nos. 09-191325, 09191326 and 09-191327, the contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an image sensing device for producing high resolution images and wide images on a single sheet by combining a plurality of digital images.
2. Description of the Related Art
Heretofore, devices have been known which combine a plurality of captured images to produce wide images or high resolution images on a single sheet, using digital image sensing devices. In a first conventional example shown in FIG. 12, a beam splitter 103 is disposed between an image sensing optical unit 101 and two two-dimensional image sensing devices 102a and 102b, so as to bifurcate the luminous flux and, for example, direct the right half of an image to the two-dimensional image sensing device 102a and direct the left half image to the two-dimensional image sensing device 102b. Digital image signals corresponding to the two images received by the two two-dimensional image sensing devices 102a and 102b are processed so as to form a single composite image.
In a second conventional example shown in FIG. 13, a single two-dimensional image sensing device 112 is fixedly attached to a so-called X-Y drive device 113 which moves the two-dimensional image sensing device 112 within an imaging area of image sensing optical unit 111, so as to receive the images of a plurality of different areas (e.g., four locations) 100a-100d of the image sensing range (object) 100, and process digital signals corresponding to each of these images to form a single composite image.
In either of the aforesaid conventional cases, when considering the object as a base, a fixed range is substantially sensed by a plurality of two-dimensional image sensing devices, thereby substantially being equivalent to increasing the number of pixels of the two-dimensional image sensing device to produce a high resolution image. On the other hand, when considering the two-dimensional image sensing device as a base, the images of the objects in different ranges are combined (i.e., pasted), thereby substantially being equivalent to widening the field angle (reducing the focal length) of image sensing optical units 101 and 111 to produce a wide image.
In the aforesaid first and second conventional examples, the image sensing optical units 101 and 111 must have a large imaging area compared to the image sensing surface (pixel region surface area) of the individual two-dimensional image sensing devices 102a, 102b, and 112, thereby increasing the size of the image sensing optical units 101 and 111 themselves. Furthermore, a large size dark box or barrel is required to house the beam splitter 103 and X-Y drive 113 and the like. As a result, the image sensing devices have the disadvantage of an overall large form factor.
Considering the image sensing optical units 101 and 111 as a base, a disadvantage arises in that only standard size images can be produced because wide images produced by combining images are substantially limited to the imaging area of the image sensing optical units 101 and 111.
In the aforesaid first and second conventional examples, disadvantages arise insofar as images are normally combined even when high resolution images are not required, thereby increasing power consumption and processing time used to combine the images.
In the second conventional example, the position of each captured image (i.e., the captured area of the target image sensing range) is dependent on the positioning accuracy of the X-Y drive device 113 because a plurality of images are captured as the two-dimensional image sensing device 112 is moved by the X-Y drive device 113. Accordingly, when the positioning accuracy of the X-Y drive device 113 is low, pixel data loss and overlap occurs in the pasted areas of the images of each region, such that the sharpness of the pasted areas in the combined image is adversely affected. Furthermore, when the image sensing device is disturbed via so-called hand vibration or the like during the movement of the two-dimensional image sensing device 112, pixel data loss and overlap are generated in a wide range near the pasted areas of each image, such that the images cannot be accurately combined, resulting in image degradation.
An object of the present invention is to eliminate the previously described disadvantages of the aforesaid conventional examples by providing a compact, light weight image sensing device capable of forming composite images at a selectable range by combining a plurality of images, and which is capable of high precision image pasting by correcting disturbances caused by hand vibration.
The aforesaid objects are attained by providing an image sensing device comprising:
a two-dimensional image sensing device which captures the image projected thereon to generate image data;
an optical unit having at least two focal lengths, for projecting an image on the image sensing device;
a focal length control unit which changes the focal length of the optical unit;
a direction control unit which controls the direction of the optical axis of the optical unit;
an image combining unit which combines a plurality of images captured by the image sensing device to form one composite image;
a mode selector which selects either a standard image sensing mode or a high resolution image sensing mode; and
an operation control unit which controls the focal length control unit, the direction control unit, the image sensing device, and the image combining unit in accordance with the image sensing mode selected by the mode selector.
When the standard image sensing mode is selected by the mode selector, the operation control unit controls the focal length control unit to set the focal length of the optical unit at a first focal length, and captures the entire target via the image sensing device. When the high resolution mode is selected by the mode selector, the operation control unit controls the focal length control unit to set the focal length of the optical unit at a second focal length longer than the first focal length, and controls the direction control unit to sequentially move the direction of the optical axis of the optical unit toward the center of each area of the target image capture range, which has been divided into a plurality of areas. The image of each area is captured via the image sensing device, and the plurality of captured images are combined via the image combining unit to produce a high resolution composite image of the entire target image.
Hence, the image sensing device of the present invention has a standard mode to capture an image at a standard resolution, and a high resolution mode to capture the image at high resolution, and is capable of switching the image sensing mode as necessary. In the standard image sensing mode, the focal length of the optical unit is set at a first focal length on the short side (i.e., the socalled wide angle side), and in the high resolution mode, the focal length of the image sensing optical unit is set at a second focal length on the long side (i.e., the so-called telephoto side). In the high resolution mode, the target image sensing range is divided into a plurality of areas, and the image of each area is enlarged to the image sensing surface area of the two-dimensional image sensing device and captured. The number of pixels of the two-dimensional image sensing device is effectively increased and a high resolution image of the target image sensing region is produced by combining the captured images.
In the high resolution mode, it is desirable that the focal length of the optical unit is initially set at the first focal length, and the entire image sensing range is captured as a reference image to subsequently form a composite image. That is, although the resolution is not high when the focal length of the optical unit is set at the first focal length on the wide angle side, a reference image without seams can be produced. A combination of images captured at the second focal length on the telephoto side of the optical unit, and particularly the process of pasting images, can be readily accomplished by referring to the aforesaid reference image.
It is desirable that the optical unit be either a bifocal lens or a zoom lens, and the second focal length be an integer multiple of the first focal length. In general, when using a bifocal lens or a zoom lens, the focal length on the so-called telephoto side is often set at an integer multiple of 2xc3x97 or 3xc3x97 of the focal length on the wide angle side. For example, when the second focal length is set at about 2xc3x97 of the first focal length, the target image sensing range is divided into four regions comprising two regions in the lateral direction and two regions in a vertical direction to produce one composite image of the entire target image. In this instance, in the high resolution mode, a resolution of about 4xc3x97 the resolution of the standard mode can be obtained. Similarly, when the second focal length is set at about 3x the first focal length, a resolution of about 9xc3x97 the resolution of the standard mode can be obtained in the high resolution mode. When considering errors in pasting each image, it is desirable that the second focal length be set somewhat shorter than an integer multiple of the first focal length.
The image sensing surface area of the two-dimensional image sensing device in the high resolution mode is desirably wider than the surface area of each image combined by the image combining means. That is, in the high resolution mode, the positions of the pasted images can be shifted to adjust for errors of the pasted areas of each image by having the image sensing surface area of the two-dimensional image sensing device wider than the surface area of each image pasted by the image combining means. Furthermore, when the image sensing device is disturbed by xe2x80x9chand vibrationxe2x80x9d after capturing a specific image and during the capturing of the next image in the high resolution image sensing mode, actual xe2x80x9chand vibrationxe2x80x9d can be corrected by shifting the positions of the pasted images while referring to the aforesaid reference image.
In the aforesaid construction, it is desirable that the direction control means is a drive mechanism for driving the housing of the image sensing units, containing the image sensing optical unit and the two-dimensional image sensing unit, in a predetermined direction to a predetermined angle. It is also desirable that the direction control means is a movable mirror disposed above the optical axis of the image sensing unit, and drivable in a predetermined direction and predetermined angle. In either case, the imaging area of the image sensing optical unit must at least cover the image sensing range of the two-dimensional image sensing device, and need not be an image sensing optical unit having an imaging area larger than the image sensing range of the two-dimensional image sensing device as in the previously described conventional examples. As a result, the overall image sensing device is lighter in weight and more compact.
As a further feature of the invention, since the image sensing device allows the target image sensing range to be optionally set via the image plane format setting unit, the direction of the optical axis of the image sensing optical unit can be controlled based on the set image plane format to capture images via a two-dimensional image sensing device and paste images via the image combining unit, to form an image of an arbitrary range having a field angle wider than the field angle of the image sensing optical unit. Therefore, the number of composite images is not limited as in the previously described conventional examples, and the composite image range (field angle) may be freely set. Not only may a user set a desired image plane format, the control of the direction of the optical axis of the image sensing optical unit and image combination may be automated.
The image plane format setting unit is preferably capable of setting the vertical and horizontal image planes at respectively identical magnification, as well as respectively different magnifications for enlargement. For example, a so-called zooming effect can be obtained by setting the magnifications of the vertical and horizontal image planes at respectively identical magnification for enlargement. Furthermore, a so-called panorama effect can be obtained by setting the magnification of the horizontal and vertical image planes at respectively different magnifications for enlargement.
The image sensing surface area of the two-dimensional image sensing device is desirably larger than the surface area of the images combined by the image combining unit. Hence, the position of the pasted images can be shifted by having the surface area of the pixel area of the two-dimensional image sensing device slightly larger than the surface area of each of the images pasted by the image combining unit, so as to allow adjustment for error in pasting the images. When the image sensing device is disturbed by hand vibration or the like after a certain image has been captured but before the next image has been captured, the position of the pasted image can be shifted to substantially correct the effects of hand vibration. The imaging area of the image sensing optical unit will cover at least the image sensing region of the two-dimensional image sensing device, so as to render unnecessary an image forming optical unit having an imaging area larger than the image sensing region of the two-dimensional image sensing device, as in the aforesaid conventional examples. As indicated above, therefore, the overall image sensing device is light weight and compact.
As a further feature, the image sensing device of the present invention captures the images near the contiguous area of at least two mutually adjacent images as a reference image and combines a plurality of captured images via the image combining unit with reference to the reference image. As a result processing of the pasted areas of the image, e.g., shifting and comparison of pixel data, can be readily accomplished. For example, when combining four images on two sheets in vertical and lateral order as in the second conventional example, the center of the target image sensing range is designated the reference image. In general, the precision of image combination can be improved by referring to a reference image because an important object is often positioned in the center of the target image sensing range. When pixel data is lost in the pasted areas due to hand vibration or the like, it is possible to correct the effects of the vibration using the pixel data of the reference image.
It is further desirable that flash illumination is used when each image is captured by the two-dimensional image sensing device. Since the emission time of the flash is extremely short, there is no vibration during the capture of a single image by the two-dimensional image sensing device.